Skip to content

Add traceable mistral and mistral3 classes#1343

Closed
anmarques wants to merge 17 commits intomainfrom
traceable_mistral3
Closed

Add traceable mistral and mistral3 classes#1343
anmarques wants to merge 17 commits intomainfrom
traceable_mistral3

Conversation

@anmarques
Copy link
Copy Markdown
Collaborator

@anmarques anmarques commented Apr 9, 2025 •
edited
Loading

SUMMARY:
This PR adds traceable versions of Mistral and Mistral3.

NOTE:
The code fails quality and style tests, but I think we should ignore them. The failures occur at the model definitions ported from transformers and keeping the changes to a minimal would help to maintain these. I added ignore commands to the headers to automatically skip linting.

TEST PLAN:

llmcompressor.trace \
    --model_id "mistralai/Mistral-Small-3.1-24B-Instruct-2503" \
    --model_class "TraceableMistral3ForConditionalGeneration" \
    --sequential_targets "MistralDecoderLayer" \
    --ignore "language_model.lm_head" "re:vision_tower.*" "re:multi_modal_projector.*" \
    --modality vision

llmcompressor.trace \
    --model_id "mistralai/Mistral-Small-3.1-24B-Instruct-2503" \
    --model_class "TraceableMistral3ForConditionalGeneration" \
    --sequential_targets "MistralDecoderLayer" \
    --ignore "language_model.lm_head" "re:vision_tower.*" "re:multi_modal_projector.*" \
    --modality text

Loading
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@kylesayrs kylesayrs Awaiting requested review from kylesayrs

@dsikka dsikka Awaiting requested review from dsikka

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@anmarques @kylesayrs @dsikka